A review of parameter learning methods based on approximate versions of the method of moments

The method of moments is an old statistical technique that fits parameters of a model by matching the average value of moments under the model with their average value over the data. Recently, approximate versions of this basic technique have been proposed in several diverse literatures including statistics, econometrics, population genetics, systems biology, ecology, epidemiology, and image texture modelling. This review connects these different approaches, addresses some key themes, and suggests several directions for future work. 1 The method of moments In this section we provide a brief introduction to the parameter learning scheme called the method of moments. Many of the subsequent methods considered in this review can be seen as approximate versions of this scheme. The goal of the method of moments is to estimate the parameters, θ, of a probabilistic model, p(y|θ), from training data, Y = {yn} N n=1. The basic idea is compute some empirical moments on the training data, Φi(Y) = 1 N PN n=1 fi(yn), and then to alter the parameters so that the expected moments under the model,